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ABSTRACT 

The current study, using a factor level analysis approach, provides evidence of the 
validity of the Social Emotional Development Instrument (SED-I) for students in higher 
education. Although there are a multitude of models and measures that borrow from the premise 
of emotional intelligence (El), very few current instruments focus exclusively on a 
developmental path of social emotional competence development for young adults. In addition, 
prior validity studies with these measures fail to include a full spectrum of potential convergent 
and discriminant analyses assessed at the factor level. This validation study suggests that the 
SED-I is a reliable, robust and comprehensive measurement for assessing college students’ social 
emotional competency changes. Limitations and suggestions for future research are also 
discussed. 
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INTRODUCTION 

Although there are a multitude of models and measures that borrow from the premise of 
emotional intelligence (El), there are no current instruments that focus exclusively on a 
developmental path of social emotional competence development for students in higher 
education. The purpose of this paper is to provide evidence of the validity of a new measure of 
social emotional competence development that has emerged in the literature (Seal, Naumann, 
Scott, & Royce-Davis, 2011a; Seal, Beauchamp, Miguel, & Scott, 2011b), called the Social 
Emotional Development Inventory (SED-I). This paper provides an empirical evidence of the 
potential utility of the new instrument, as well as an in-depth analysis of the relationships with 
various emotional/social assessments. There are few validity studies that have compared multiple 
El measures with other assessments and even fewer have examined constructs at the factor and 
sub-factor levels. Previous studies typically fail to examine constructs at the factor level, often 
defaulting to comparing total scores with one another. Therefore, a key contribution of the 
current paper is the integration of multiple measures focusing on factor level analysis, within a 
pre-defined framework of a priori hypotheses. The paper demonstrates the valid use of the SED-I 
self-report assessment as one of a series of developmental tools available for educators in higher 
education to better assess student knowledge, traits and behaviors. To accomplish the goal of 
providing validation for the SED-I, this paper provides an overview of social emotional 
competence development, the related soft skills measures, and a series of sub-studies that 
examine the convergent and discriminant relationships between the SED-I and relevant, related 
instruments. 

Emotional Intelligence 

In general, El may be defined as the overlap between emotion and intelligence, or more 
simply, the intelligent use of emotions. El is the overall term reflecting the intelligent use of 
emotions, emotional quotient (EQ) refers specifically to the emotional traits as advocated by Bar- 
On (1988), emotional ability (EA) refers to the underlying potential of El as advocated by 
Salovey and Mayer (1990), and emotional competency (EC) refers to the El skills related to 
superior performance as advocated by Goleman (1995; 1998). Specifically, El may be defined as 
a constellation of capacities, patterns, and behaviors to recognize and regulate the emotions of 
self and others toward successful environmental adaptation (Seal & Andrews-Brown, 2010). 

Social Emotional Competence Development 

Social emotional competence development (SECD) emerged from the integration of 
social and emotional intelligence theories toward the practice of student development in higher 
education. The model is primarily an outgrowth of the emotional social competence model 
developed by Boyatzis and Goleman (Boyatzis, Stubbs, & Taylor, 2002; Goleman, Boyatzis, & 
McKee, 2002). Their model was derived heavily from the Self-Assessment Questionnaire 
(SAQ), developed by Boyatzis (1994) which focuses on business managers and MBA students. 

In contrast, the SED-I items were developed using undergraduate and graduate level focus 
groups geared toward tapping underlying competences ascribed to college academic, relational, 
and career success (Seal et al., 2011a). 
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SECD is defined as the enhancement of personal capacity to manage environmental 
challenges. The concept is a consequence of the application of social intelligence, emotional 
intelligence, and competence research toward higher education development. The conceptual 
model encompasses four primary factors (Seal et al., 2011a): Self-awareness (knowledge and 
understanding of your emotions and aptitudes), Consideration of Others (regard for the person 
and situation before thinking and acting), Connection to Others (ease and effort in developing 
rapport and closeness with others), and Influence Orientation (propensity to seek leadership 
opportunities and move others toward change). The assumption of SECD is that students who 
develop their capacity to understand themselves, consider the world around them, build 
meaningful relationships, and foster positive changes will have an advantage in meeting 
academic, relational, and career challenges (Kuh, Kinzie, Schuh, Whit, & Associates, 2010). 

The model and measure are best used in conjunction with other developmental tools to assist in 
identifying key strengths and limitations of students. 

The SECD model is measured using the SED-I which includes 48 self-reported Likert 
style questions asking the test-taker to indicate how often a particular question relates to them (1 
= very rarely; 7 = very often). Each of the four factors is linked to 12 of the questions on the 
survey instrument. In assessing Self-awareness and Consideration of Others, the question is 
asked “how often that statement is true of you,” while Connection to Others and Influence 
Orientation factors use the prompt “predict how often your friends would say that statement is 
true of you.” The SED-I demonstrated adequate internal consistency (r = .91) with a university 
population of freshman (Seal et al., 2011b). However, the instrument needs to be assessed 
against other, similar types of instruments using an expanded sample population in order to 
validate the tool. 

Alternative Measures of Soft Skills 

One of the major concerns regarding measures of soft skills (e.g., skills that focus on 
relationships between people), particularity those that have emerged from the literature on El, is 
the lack of robust, comprehensive validity studies that incorporate multiple measures of El, as 
well as personality and other related constructs, and examine instruments at the factor level 
(rather than just comparing total scores).To address those issues, the present validity study uses 
two prominent measures of El as well as measures of personality, emotions, self-monitoring, and 
social desirability. These six soft s ki lls assessments are leading measures in the fields of 
psychology, education, and management. 

Mayer-Salovey-Caruso Emotional Intelligence Test 2.0 (MSCEIT 2.0) 

Salovey and Mayer (1990) defined El as “the subset of social intelligence that involves 
the ability to monitor one’s own and others’ feelings and emotions, to discriminate among them 
and to use this information to guide one’s thinking and actions” (p. 189). Their version of El 
emerged from the literature on social intelligence with a focus on how emotions facilitate 
thought using an instrument that had right and wrong answers (using consensual and/or expert, 
normed scoring). Generally, emotional ability is the potential capacity to understand and explain 
emotions and to use that knowledge to improve decision making. 

The MSCEIT uses four branch scores: (1) Perceiving Emotions indicates the degree to 
which the respondent can identify emotion in himself or herself and others; (2) Facilitating 
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Thought specifies the degree to which the respondent can use his or her emotions to improve 
thinking; (3) Understanding Emotions suggests how well the respondent understands the 
complexity of emotional meanings; and (4) Emotional Management registers how well the 
respondent is able to manage emotions in his or her own life and in others. Some overlap 
between El measures would be expected. However, since the underlying assumptions and 
method of assessment are different, minimal relationships between the MSCEIT and SED-I are 
expected surrounding the Emotional Management scores. 

Bar-On Emotional Quotient Inventory (EQ-i) 

Bar-On (2006) defined El as the “cross section of interrelated emotional and social 
competencies, skills, and facilitators that determine how effectively we understand and express 
ourselves, understand others, and relate with them, and cope with daily demands” (p. 14). This 
version of El emerged from the literature on positive psychology with a focus on psychological 
well-being using a self-report measure that included normed scoring. Overall, emotional quotient 
involves the preferred emotional patterns that influence understanding and coping with 
environmental demands to achieve psychological well-being. 

The EQ-i uses five component scores: (1) Intrapersonal EQ which includes emotional 
self-awareness, assertiveness, self-regard, self-actualization, and independence; (2) Interpersonal 
EQ which includes interpersonal relationships, empathy, and social responsibility; (3) Stress 
Management EQ which includes stress tolerance and impulse control; (4) Adaptability EQ which 
includes problem solving, reality testing, and flexibility; and (5) General Mood EQ which 
includes happiness and optimism. Given the overlap in assumptions and method between the 
SED-I and EQ-i, we would assume a greater level of statistical overlap than with the MSCEIT, 
particularly in the areas of Inter- and Intrapersonal EQ and SECD. 

International Personality Item Pool (IPIP) Big-Five Factor Markers 

Personality is generally defined as a stable set of characteristics and tendencies that 
determine those commonalities and differences in the psychological behavior (thoughts, feelings, 
and actions) of people that have continuity in time and that may not be easily understood as the 
sole result of the social and biological pressures of the moment (Goldberg, 1992). 

To assess personality, the IPIP focuses on the Big Five factor model of (1) Extraversion - 
engagement with the external world; (2) Agreeableness - cooperation and social harmony; (3) 
Conscientiousness - control, regulation, and direction of impulses; (4) Emotional Stability - 
coping effectively with normal demands of life; and (5) Intellect - novelty, variety, change, and 
creativity (Goldberg, 2001). As with all self-report instruments that focus on social skills, we 
would expect a high level of method bias as well as significant relationships between the SED-I 
and IPIP, in particular the factors of Extraversion, Agreeableness, and Conscientiousness. 

Trait Meta-Mood Scale (TMMS) 

An early pre-cursor to the modem El measures, the TMMS (Salovey, Mayer, Goldman, 
Turvey, & Palfai, 1995) is focused on Emotional Attention (attending to thoughts and feelings), 
Emotional Clarity (distinguishing among feelings), and Emotional Repair (improving negative 
moods). As a measure of emotional self-awareness we would expect an overlap between the 
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TMMS scales of Emotional Attention and Emotional Clarity and the SED-I Self-Awareness 
factor. 

Self-Monitoring Scale (SM) 

Self-monitoring is defined as the presumed and consistent patterns of individual 
differences in the extent to which people regulate their self-presentation by tailoring their actions 
in accordance with immediate situational cues (Lennox & Wolfe, 1984). In particular self- 
monitoring is divided into two aspects: (1) Modify Self - ability to modify self-presentation and 
(2) Sensitivity Others - sensitivity to the expressive behavior of others. In the article introducing 
the social emotional competence development model, Seal et al. (2011a) defined Consideration 
of Others as the thoughtful regard for the person and situation in anticipating the likely 
consequences before thinking and acting. One of the components of Consideration is 
monitoring, which includes recognizing/anticipating consequences and regulating/thinking 
before speaking or acting. Since the Consideration of Others factor of the SED-I specifically 
relates to monitoring one’s behavior in the presence of others, we would expect a significant 
positive relationship between SM and the Consideration factor score. 

Marlowe-Crowne Social Desirability Scale (SD) 

Social desirability is the tendency of respondents to reply in a manner that will be viewed 
favorably by others (Reynolds, 1982). Since the SED-I is a self-report instrument, the question of 
which factors are influenced by social desirability becomes a concern. The hope is that none of 
the factors indicates a significant relationship to social desirability. 

RESEARCH DESIGN 

The six critical measures reviewed above establish the framework for the factor analysis 
reported here. The current paper builds on the psychometric assessment paper (Seal et al., 2011b) 
by expanding the demographics of the initial study, considering students at different levels in 
their education (e.g., undergraduate, graduate students), establishing test-retest reliability, as well 
as examining convergent/discriminant validity evidence (e.g., the relationship of the SED-I to 
alternative emotional intelligence and personality instruments). Seal et al. (2011b) previously 
assessed other aspects of the measure, including item generation, internal consistency, factor 
structure, and readability grade level. In the current investigation, three assessment studies to 
evaluate the SED-I measures were conducted, including psychometrics assessment, test-retest 
reliability assessment, and convergent/discriminant validity assessment 

STUDY 1: PSYCHOMETRICS 

To assess the generalizability of the SED-I instrument, a series of psychometric analyses 
were conducted to replicate results from the freshman study (Seal et al., 2011b), using transfer, 
professional, and graduate students. 
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METHOD 

A total of 1,134 students completed the survey. In order to test the reliability of the 
instrument, students were divided into two different groups based on student characteristics. 
Group 1 includes 632 first time freshman students from the original study (Seal et al., 2011b) and 
Group 2 included 502 students comprised of sophomores, juniors, seniors, transfer students, 
professional students, and graduate students. 

Group 1 was made up of 62% females and 38% males, between the ages of 16 and 19 
years old, with a mean age of 18. The ethnic makeup of the sample was 39% White/Non- 
Hispanic, 35% Asian/Pacific Islander, 15% Hispanic, and 11% were comprised of African 
American, Native American, multiethnic, and international/unknown/other categories. 

Group 2 was comprised of 60% female and 40% male, between the ages of 17 and 53, 
with a mean age of 23.The ethnic makeup of the sample was 44% Asian/Pacific Islander, 32% 
White/Non-Hispanic, 9% Hispanic, and 15% comprised of African American, Native American, 
Multi-ethnic, and intemational/unknown/other categories. 

Participants were recruited through an email campaign asking students to complete the 
SED-I measure online receiving a web link to the survey in the email. Once students clicked on 
the email link, they were given an informed consent screen with the option of opting into the 
study. Total time to complete the measure was about 15 minutes. 

Results 

The psychometric analyses included descriptive statistics, data screening, internal 
consistency assessment, and exploratory factor analysis. 

The descriptive statistics for item scores, item-factor correlations, 4-factor 
communalities, and principal components pattern matrices coefficients (using direct oblimin 
rotation, 8 = 0) are presented in Table 1. See Table 1 (Appendix). 

Most items were significantly (p < .001) negatively skewed (31) and leptokurtic (11). 
Scores were reflected and transformed to square root equivalents. As with the Group 1 freshman 
data set (Seal et al., 2011b), the square root transformation reduced the number of skewed (27) 
and leptokurtic cases (3). Using z score > 4.00 as the criterion, one univariate outlier was 
detected and removed. The search for multivariate outliers was limited to those cases with 
complete data (517). Fifteen people met the outlier criterion of a Mahalanobis distance greater 
than 103 (p < .00001) and these scores were removed. The resulting data set consisted of the 502 
cases indicated above. 

The untransformed scores were used to assess internal consistency (total score and factor 
reliability) using Cronbach’s coefficient alpha. The total score alpha of .91 indicates good 
internal consistency. As compared to the results reported by Seal et al., (2011b), this sample had 
slightly higher internal consistency scores across the factors, except for the Influence Orientation 
factor. Table 2 contains the coefficient alpha values for this study and the study by Seal et al. 
(2011b). See Table 2 (Appendix). 

To determine the number of factors, a principle components analysis (PCA) was 
conducted with the untransformed data. Indicating that the correlation matrix was factorable, the 
Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy was .89. Seal et al. (2011) reported 
.90 for their freshmen data set. The PCA, with SPSS default settings in place, was conducted on 
the 48 items using the raw data and 10 factors were extracted. The resultant information is 
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returned in the form of a scree plot and variance tables. The scree plot showed elbows between 
components five and six, leveling afterward. The drop between factors five and six shows where 
the eigenvalues were < 2. The eigenvalue on the fifth factor was 2.20; however, it was still 
dropped to preserve the 4-factor model. The PCA was run again, forcing four factors; see Table 1 
for communalities and pattern matrix coefficients. Multiple PCAs were conducted with two data 
transformations and several rotation procedures; the results were not substantially improved 
relative to the results from the raw data analysis reported here. The four-factor solution 
accounted for 40% of the variance in the dataset; the same procedure accounted for 38% of the 
variance for the Seal et al. (2011) data set. Thus, there was little variance in results across the 
two data sets. 

STUDY 2: TEST-RETEST RELIABILITY ASSESSMENT OF THE SED-1 

To assess the reliability of the SED-I instrument, a test-retest study was conducted to 
determine the consistency of scores over time. 

Method 

A total of 84 freshmen participants completed the SED-I twice within a 3 to 7 month time 
period with the majority of the students completing the two self-assessments within a 4 to 6 
month interval, admittedly a long delay for self-report, test-retest reliability assessment of a 
developmental instrument. Students were recruited through their classes to re-take the SED-I 
instrument for the test-retest study. 

Results 

Pearson test-retest correlation coefficients were calculated for each of the 48 items and 
for the four factors; the factor results are reported along with two-tailed probabilities that the 
correlation values exceed 0. Means and standard deviations were computed for each set of test 
scores for each factor. With regard to the four factors, the results show good test-retest 
consistency for the Connection with Others and Influence Orientation factors, but not for the 
other two factors as outlined in Table 3. See Table 3 (Appendix). 

The means declined for every factor. Given that unpredicted result, post-hoc /-tests were 
calculated for the mean differences for each factor to assess the reliability of the score 
reductions. None of the 95% confidence intervals included zero, that result in addition to the 
result that no /-test value showed a probability greater than .01, suggests that the score reductions 
for each factor were reliable. The Connection to Others factor showed the least amount of 
reduction. The reliability and mean differences data imply that during freshmen year, students 
tend to modify their Self-Awareness and Consideration of Others, showing a trend of reducing 
their self-ratings in these areas. Results of the /-tests are presented in Table 4. See Table 4 
(Appendix). 

STUDY 3: CONVERGENT/DISCRIMINANT VALIDITY TESTING 

To provide evidence of the potential validity of the SED-I instrument, a series of 
convergent and discriminant studies were conducted, comparing the SED-I to other general 
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measures of El (MSCEIT and EQ-i) as well as personality (IPIP), emotion (TMMS), self¬ 
monitoring (Lennox & Wolfe, 1984), and social desirability (Reynolds, 1982). 

Method 

The measures chosen for inclusion are among the most widely used measures in 
psychology, education, and management. For the purposes of this study, we used several 
decision rules to define the level of relationship (correlation), criterion for convergent validity 
(alpha level to reject the null hypothesis), and criterion for discriminant validity (alpha level to 
accept the null hypothesis). In terms of correlations (level of relationship of the SED-I factors to 
the alternative measure), we created the following categories: a correlation of less than ± 0.09 is 
negligible, ± 0.10 through 0.30 is a small correlation, ± 0.31 through 0.49 is a moderate 
correlation, and > ± 0.50 is a strong correlation. In terms of convergent validity, the alpha level 
was set at the standard .05 to minimize type I errors. Finally, for discriminant validity, the alpha 
level was set at a higher threshold of .20 to minimize type II errors. 

In order to establish the a priori hypotheses, a series of steps were followed. Two 
independent researchers predicted the level of convergence between the SED-I and the various 
measures based on (a) the purpose and definitions of each of the measures as indicated by their 
authors and (b) a language analysis that assessed the frequency of exact match, close match, and 
concept match. First the researchers mapped out conceptual overlaps that occurred between 
instruments. Any definitions of constructs that appeared related were marked, regardless of 
actual method. Second, a set of researchers familiar with the instruments then reviewed the 
predicted relationships and adjusted them based on intent of the construct as well as the 
operational method used. The researchers examined each assessment at the factor level and 
predicted a negligible, small, moderate, or strong correlative value with SED-I factors. The 
predictions, both summative and semantic, were then reviewed by a third researcher, who 
synthesized the results matrix into one set of predictions. The two-pronged approach attempted 
to develop a comprehensive view of the SED-I convergent/discriminant validity. The final 
hypotheses were then compared to observed Pearson’s correlation coefficients; the bivariate 
correlation was run between each assessment’s factors and the SED-I factors with the two-tailed 
test of significance, mean, and standard deviation reported and significant correlations flagged. 
Correlations were declared significant at the .05 level for convergent validity and at the .20 level 
for discriminant validity. 

The following details the six sub-studies, comparing the specific instrument scores to the 
SED-I scores. In each sub-study, we indicate the participants (as each had a different sub-set of 
the overall sample), hypothesis (established through the process outlined above), and results (of 
the Pearson correlation coefficient). For an overview of the results, see Table 5 (Appendix). 

Study 3.1: Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT) 

As noted earlier, Salovey and Mayer (1990) defined El as the capacity to understand and 
explain emotions and to use that knowledge to improve decision making. Their definition 
focused on differentiating individual levels of specific emotional reasoning capacities using the 
Mayer-Salovey-Caruso Emotional Intelligence Test or MSCEIT (Mayer, Salovey, & Caruso, 
2002) to measure those differences. The MSCEIT uses right/wrong answer formats (based on 
consensual and expert scoring) that are normed using demographic information (age, gender, and 
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ethnicity) to differentiate levels of emotional ability. The MSCEIT uses a hierarchical model of 
related abilities that reflect overall El potential capacity by measuring four branch scores 
(Perceiving Emotions, Facilitating Thought, Understanding Emotions, and Emotional 
Management), two area scores (Experiential Emotional Intelligence and Strategic Emotional 
Intelligence) and one final total score (Emotional Intelligence). For our purposes, we 
administered the MSCEIT online, using consensual scoring, and focused on the four branch 
scores. 

The MSCEIT sub-study included 85 participants. Fifty-one (60%) were female, 32 (38%) 
were male, and the remainder were not identified. Participants’ age ranged from 17 to 53, (mean 
= 21, SD = 6.3). Participants were enrolled in the undergraduate pharmacy/health program 
(38%), the graduate education program (32%), various undergraduate programs in arts and 
sciences (19%) and the remainder in an aggregate of other schools with less than 5% each (21%). 
Our initial hypotheses were the following: 

H3.1.1 - The Managing Emotions branch of the MSCEIT would have a small 
significant positive correlation with the Self-awareness factor score of the SED-I. 

H3.1.2 - The Managing Emotions branch of the MSCEIT would have a small 
significant positive correlation with the Influence Orientation factor of the SED-I 
H3.1.3 - Since the underlying assumptions and methods of assessment are 
different, we would expect that the two measures would not correlate in other areas, that 
is, the remaining branch scores of the MSCEIT and factor scores of the SED-I should not 
be significantly correlated. 

To test the hypotheses, Pearson’s correlation coefficients were calculated between each 
branch of the MSCEIT and each factor of the SED-I (Table 6). 

Hypothesis 3.1.1 was not supported as there was a negligible correlation (-.03) that was 
not statistically significant (even at the alpha = .20 level), indicating no relationship between the 
variables. 

Hypothesis 3.1.2 was not supported as there was a small, negative correlation (-.19) that 
was not statistically significant at the .05 level, but was significant at the .20 level, indicating a 
possible negative relationship between the variables. 

Hypothesis 3.1.3 was largely confirmed across factors of both instruments; however, 
there was a small correlation between Perceiving Emotions (MSCEIT) and Consideration of 
Others (SED-I) as well as a small correlation between Using Emotions (MSCEIT) and Influence 
Orientation (SED-I), neither of which met the alpha > .20 decision rule, indicating possible 
relationships between the variables. 

Study 3.2: Bar-On Emotional Quotient Inventory (EQ-i) 

As noted earlier, Bar-On (2006) defined El as the emotional patterns that influence 
understanding and coping with environmental demands to achieve psychological well-being. His 
definition focuses on the emotional-social traits or patterns that contribute toward effective 
psychological functioning using the Emotional Quotient Inventory or EQ-i (Bar-On, 1997) to 
measure the self-reported patterns. The EQ-i is a self-report measure of emotional quotient that 
uses a compensatory model of 5 composite scales (Intrapersonal, Interpersonal, Stress 
Management, Adaptability, and General Mood) comprised of 15 sub-scales. For our purposes, 
we administered the EQ-i online and focused on the five composite scale scores. 
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The EQ-i sub-study included 280 participants, 168 (60%) of which were female and 110 
(38%) of which were male, and 2 not identified. Participants’ ages were between 17 and 53, 
(mean = 20, SD 4.3). Participants were undergraduate and graduate students, enrolled in business 
administration (45%), pharmacy/health programs (24%), arts and science (16%), education 
(10%), and the remainder in an aggregate of other schools (5%). 

Our initial hypotheses were as follows: 

H3.2.1 - The Intrapersonal composite score of the EQ-i would have a moderate 
significant positive correlation with the Self-awareness factor score of the SED-I. 

H3.2.2 - The Interpersonal composite score of the EQ-i would have a moderate 
significant positive correlation with the Consideration of Others factor score of the SED- 
I. 

H3.2.3 - The Interpersonal composite score of the EQ-i would have a moderate 
significant positive correlation with the Connection to Others factor score of the SED-I. 

H2.4 - Stress Management, Adaptability, and General Mood of the EQ-i would 
have a small, but not significant relationship with the SED-I factors. 

To test the hypotheses Pearson’s correlation coefficients were calculated between each 
composite scale of the EQ-i and each factor of the SED-I (Table 7). 

Hypothesis 3.2.1 was supported, as there was a moderate, significant positive relationship 
between the Intrapersonal composite score of the EQ-i and the Self-awareness factor score of the 
SED-I. In addition, although there were other significant relationships, this relationship had the 
highest correlation between EQ-i scores and Self-awareness. 

Hypotheses 3.2.2 and 3.2.3 were partially supported, as there was a significant positive 
correlation between the Interpersonal composite score of the EQ-i and the Connection to Others 
and Consideration of Others factors of the SED-I. However, the correlations were small instead 
of the predicted moderate values, and the other factors (Self-awareness and Influence 
Orientation) displayed unpredicted, but similar small correlations. 

Hypothesis 3.2.4 was not supported, as across all the analyses there were small to 
moderate positive significant correlations between the measures, indicating substantial overlap 
between the two instruments. 

Study 3.3: International Personality Item Pool (IPIP Big 5) 

Personality is generally defined as a stable set of characteristics and tendencies that 
determine those commonalities and differences in the psychological behavior of people that have 
continuity in time (Goldberg, 1992). Personality was measured using the International 
Personality Item Pool (Goldberg, 2001). The IPIP assesses the Five-Factor Model (FFM) of 
personality, originally developed by Thurstone (1934) and clarified by Goldberg (1992) and is 
considered “the central paradigm for defining and measuring personality” (Polednik & Greig, 
2000, p. 9). The IPIP inventory uses 50 items and users rate how accurately the phrases 
describing people’s behavior describes them on a five-point Likert scale (from “1” very 
inaccurate to “5” very accurate), and the measure takes approximately 10-20 min to complete. 
The alpha reliability coefficients of the items range from .71 (Openness to Experience) to .89 
(Extroversion); (Buchanan, Johnson, and Goldberg, 2005). 

The IPIP sub-study included 181 undergraduate and graduate participants: 93 (51%) were 
female and 88 (49%) were male. Participants were between 17 and 44 with a mean of 19 and a 
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standard deviation of 3.0. Participants were enrolled in business administration (84%), arts and 
sciences (12%), and the remainder is an aggregate of other schools (4%). 

Our initial hypotheses were the following: 

H3.3.1 - Extraversion and Agreeableness scale scores of the IPIP will have small 
positive correlations with the Connection to Others factor score of the SED-I. 

H3.3.2 - Conscientiousness scale scores of the IPIP will have a small positive 
correlation with the Influence Orientation factor score of the SED-I. 

H3.3.3 - None of the other IPIP scale scores (including Emotional Stability and 
Intellect) will have significant positive correlations with any of the SED-I factor scores. 
To test the hypotheses Pearson’s correlation coefficients were calculated between each 
factor of the IPIP and the factors of the SED-I (Table 8). 

Hypothesis 3.3.1 was supported, as there was a strong positive correlation between the 
Connection to Others factor of the SED-I and the Extraversion scale score of the IPIP and a 
moderate positive correlation between the Agreeableness scale score of the IPIP. In addition, 
those were the two highest correlations between the Connection to Others factor and the other 
IPIP scores. 

Hypothesis 3.3.2 was not supported, as there was a nonsignificant negligible 
Conscientiousness-Influence correlation. 

Hypothesis 3.3.3 was not supported, as both Emotional Stability and Intellect had small 
to moderate significant positive correlations with each factor of the SED-I. In addition 
Extraversion had moderate to strong correlations with Self-awareness, Influence Orientation, and 
Connection to Others; Agreeableness had small to moderate relationships with all SED-I factors; 
and Conscientiousness had small to moderate correlations with Connection to Others, Self- 
awareness, and Consideration of Others. Overall, there is quite a bit of overlap between the IPIP 
and SED-I. 

Study 3.4: Trait Meta-Mood Scale (TMMS) 

The Trait Meta-Mood Scale (TMMS) focuses on assessing three core areas of emotion: 
(1) Attention - attention that individuals devote to their feelings; (2) Clarity - clarity of their 
experiences of these feelings; and (3) Repair - their beliefs about terminating negative mood 
states and prolonging positive ones (Salovey et al., 1995, p. 127). The TMMS is a short, 30-item 
assessment that uses a five-point Likert scale asking participants how strongly they agree or 
disagree with each statement. Results are organized into one of three factors: Attention, Clarity, 
or Repair. 

The TMMS sub-study included 248 undergraduate and graduate participants; 152 (61%) 
were female and 94 (38%) were male, with 1% not assigned. Participants were between 17 and 
53 where M = 19 and SD = 3.3. Participants were enrolled in business administration (43%), 
pharmacy/health sciences (36%), arts and sciences (10%), education (7%), and the remainder in 
an aggregate of other schools (4%). 

Our initial hypotheses were as follows: 

H3.4.1 - Attention to feelings and Clarity of feelings scale scores from the 
TMMS will have moderate positive correlations with the Self-awareness factor score of 
the SED-I. 

H3.4.2 - Repair scale score should not correlate with any of the SED-I factor 

scores. 
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H3.4.3 - Attention to feelings and Clarity of feelings scale scores will have 
significant positive correlations with Consideration, Connection, or Influence factors 
scores of the SED-I. 

To test the hypotheses Pearson’s correlation coefficients were calculated between each 
factor of the TMMS and factor of the SED-I (Table 9). 

Hypothesis 3.4.1 was partially supported, as there was a strong positive correlation 
between Clarity of feelings of the TMMS and Self-awareness of the SED-I. However, the 
relationship between Attention to feelings and Self-awareness was moderate and was the weakest 
correlation, as Influence Orientation, Consideration of Others, and, in particular, Connection to 
Others all had stronger correlations. 

Hypothesis 3.4.2 was not supported as Repair emotions of the TMMS had a moderate 
positive significant relationship to each factor of the SED-I. Of note is the consistency of the 
relationship between Repair and the various SED factors. 

Hypothesis 3.4.3 was not supported as Attention to feelings and Clarity of feelings had 
significant moderate positive relationships to each factor of the SED-I as indicated above. 

Study 3.5: Self-Monitoring (SM) 

Self-monitoring presumes consistent patterns of individual differences in the extent to 
which people regulate their self-presentation by tailoring their actions in accordance with 
immediate situational cues (Lennox & Wolfe, 1984, p. 1349). As mentioned earlier, the SED-I 
dimension of Consideration of Others is the thoughtful regard for the person and situation in 
anticipating the likely consequences before thinking and acting. One of the stated components of 
Consideration is monitoring, which includes recognizing/anticipating consequences and 
regulating/thinking before speaking or acting. Thus, we would expect a significant, positive 
relationship between Self-monitoring and the Consideration of Others factor of the SED-I. 

The Self-Monitoring assessment (SM) uses a 13-item survey, asking participants to 
respond to 5-point Likert type scale regarding their level of agreement with each item. Scores are 
organized into two key dimensions of self-monitoring: (1) Modify self - ability to modify self¬ 
presentation and (2) Sensitivity Others - sensitivity to expressive behavior of others. 

The SM sub-study included 209 undergraduate and graduate participants; 118 (57%) 
were female and 89 (43%) were male. Participants were between 17 and 53 with a mean of 20 
and a standard deviation of 4.2. Participants were enrolled in business administration (68%), arts 
and sciences (17%), education (8%), and the remainder in an aggregate of other schools (7%). 
Our initial hypotheses were the following: 

H3.5.1 - Modify Self and Sensitivity Others of SM should both have a moderate 
significant positive correlation to the Consideration of Others factor of the SED-I. 

H5.2 - Neither of the SM scale scores should have a moderate significant 
relationship to the three other SED factors. 

To test the hypotheses Pearson’s correlation coefficients were calculated between each 
dimension of the SD and factor of the SED-I (Table 10). 

Hypothesis 3.5.1 was partially supported in that both modify self-presentation and 
sensitivity to others of the SM had significant, positive relationships with Consideration of 
Others of the SED-I; however the correlations were small, not moderate. 
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Hypothesis 3.5.2 was partially supported, as there was a small significant relationship 
between Self-awareness on the SED-I and sensitivity to others on SM; however, the other 
relationships corresponded as hypothesized. 

Study 3.6: Marlowe-Crowne Social Desirability Scale (SD) 

Social desirability is the tendency for individuals to respond in a manner that will be 
viewed favorably by others (Reynolds, 1982). Correlations with this scale score could indicate 
potential for bias due to the pressure on respondents to appear socially desirable. The scale is a 
short, 13 item survey that uses true/false answers. 

The SD sub-study included 239 participants, 147 (62%) were female and 90 (38%) were 
male. Participants were between 17 and 53 years of age, (mean = 19, SD = 3.5). Participants 
were enrolled in business administration (45%), pharmacy/health (33%), arts and science (10%), 
education (8%), and the remainder is an aggregate of other schools (7%). 

Our initial hypothesis was: 

H3.6.1 - The four factor scores of the SED-I should not have significant 

relationships with the SD score, and any correlations should be negligible. 

To test the hypotheses Pearson’s correlation coefficients were calculated between each 
factor of the SED-I and SD (Table 11). 

Hypothesis 3.2.1 was partially supported in that Self-awareness, Connection to Others, 
and Influence Orientation of the SED-I all had negligible correlations with SD and all failed to 
meet the alpha .20 decision rules. However, Consideration of Others on the SED-I exhibited a 
small significant positive correlation to SD. 

DISCUSSION 

The current study builds on the findings of earlier research (Seal et al., 201 lb) that 
validates a measure of social emotional competence development for higher education students, 
the SED-I. This robust and comprehensive measure is expected to help educators to better assess 
college students’ knowledge, traits, and behaviors to further their self-development. Although a 
large number of El measures exist, there are few current instruments that focus exclusively on 
this constituent group. In addition, prior validity studies with emotional competency measures 
have lacked a full spectrum of potential convergent/discriminant analyses assessed at the factor 
level. 

The psychometric properties of a psychological test should be assessed from multiple 
occasions of measurement (McArdle & Woodcock, 1997). The principal components analysis 
results and the internal consistency of the SED-I were generally positive and similar to the 
results of the previous study by Seal et al. (2011b). The current data set exhibited slightly higher 
internal consistency scores across all factors, except the Influence factor. A 4-factor model was 
retained. 

The SED-I factors of Influence Orientation and Connection to Others exhibited higher 
test-retest reliability as compared to the Self-awareness and Consideration of Others factors. 

This may indicate that the Influence Orientation and Connection to Others factors are more 
stable over time. These findings are not surprising given that measures of Self-awareness and 
Consideration of Others might be expected to change over relatively brief time periods for 
collegiate students and thus demonstrate lower test-retest correlations. 
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The correlations between the various measures and the SED-I can be seen as a successful 
integration of the concepts that create one picture of development possibilities. Each measure 
examines separate elements of social and emotional competency development with the exception 
of the Crowne-Marlow Social Desirability scale. The SED-I takes a unique approach toward 
applying these elements to practical use for higher education. While we would expect some 
overlap and correlations to exist, we did not find a measure that correlated so strongly with the 
SED-I as to suggest the SED-I is an unnecessarily redundant measure. Our a priori hypotheses 
were grounded in multiple approaches to construct understanding and our data were measured at 
more than one point in time providing more confidence in the findings. 

Limitations 

As with any study, several caveats should be considered in the interpretation of the 
results of our study. First, the test-retest component of the study had a longer time frame than is 
generally used to examine the psychometric properties of a psychological test. However, the fact 
that the Connection to Others and Influence Orientation factors of the measure demonstrated 
high test-retest reliability even after a significant time lag suggests these dimensions appear to be 
robust. Second, our results may have been affected by same source bias or common method 
variance (CMV) which has been defined as “systematic error variance shared among variables 
measured with and introduced as a function of the same method and/or source” (Richardson, 
Simmering, & Sturman, 2009, p. 763). Because all of the measures in the current study were 
self-reported, the items all came from the same source and, thus, may be subject to CMV. Third, 
we did not explicitly measure the Influence Orientation factor by any assessment other than the 
SED-I. Thus, the correlative value of the SED-I Influence Orientation factor could not be 
comparatively assessed in this analysis. Finally, due to the challenges of obtaining research 
participants on a college campus, different sampling procedures were used for each of the 
different sub-studies in this study, with varying subsets of students completing varying 
combinations of the measures. 

Future Directions 

Future studies will need to provide additional reliability and validity evidence for the 
SED-I. Based on the results of this study, specifically the PCA results for several questions and 
the test-retest reliability study, some items may need to be revised or deleted from the measure. 
Although the challenges will remain in recruiting college-aged students, additional studies 
should be employed with a shorter time lag design and across a variety of grade and/or age levels 
to help uncover outside factors explaining the differences in the test-retest scores over time. In 
the future, a confirmatory factor analysis will need to be conducted to validate the factor 
structure of the measure. Additionally, future research should include a measure that is 
comparative to the Influence Orientation factor on the SED-I to allow for convergent validity 
testing. Because our data is limited to self-report data, future research should gather data from 
multiple non-self-report sources, such as reports by others or behavioral data. Objective outcome 
measures that would be especially relevant to this area of research include grades, retention, 
teamwork, relationship quality, or career readiness. Lastly, understanding the predictive utility of 
this measure will be important and encourage greater use of the measure. This measure could 
show added value as if it demonstrated the ability to predict grades and college retention, as well 
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as show career or relationship success. Although the potential benefits of social emotional 
competence have been demonstrated (Boyatzis, Stubbs, & Taylor, 2002; Seal, Boyatzis, & 

Bailey, 2006) previously, the focus has been on an adult population. The SED-I is an assessment 
that is appropriate for college-aged students accounting for the unique developmental stage and 
experience. 
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Table 1-Item Scores, Correlation, Communalities, and Coe 


■ficients 


Item 

Mean 

Median 

S.D. 

r - Factor 

Com. 

4 Factor 

Aware 

Consider 

Connect 

Influence 

Q01 

5.52 

6 

1.10 

.52 

.41 

.41 


.33 


Q02 

4.83 

5 

1.20 

.71 

.67 




-.84 

Q03 

4.45 

4 

1.40 

.46 

.45 



.66 


Q04 

4.70 

5 

1.18 

.74 

.73 




-.88 

Q05 

5.12 

5 

1.20 

.52 

.39 



.36 

-.40 

Q06 

5.01 

5 

1.02 

.59 

.46 




-.64 

Q07 

5.33 

6 

1.27 

.48 

.40 




-.44 

Q08 

4.74 

5 

1.03 

.54 

.40 




-.57 

Q09 

5.24 

5 

1.28 

.46 

.31 



.44 


Q10 

4.79 

5 

1.38 

.39 

.24 




-.48 

Qll 

5.81 

6 

1.02 

.56 

.44 

.43 


.40 


Q12 

4.75 

5 

1.12 

.62 

.53 




-.75 

Q13 

4.46 

4 

1.42 

.43 

.44 



.65 


Q14 

4.46 

4 

1.23 

.69 

.63 




-.82 

Q15 

5.32 

5 

1.25 

.54 

.46 

.34 


.50 


Q16 

4.84 

5 

1.19 

.34 

.23 





Q17 

4.58 

5 

1.30 


28 

—4J— 


.34 

.56 


Q18 

3.81 

4 

1.37 


50 

.39 




-.65 

Q19 

4.88 

5 

1.22 


52 

.39 

.35 


.36 


Q20 

4.74 

5 

.99 


32 

.18 




-.35 

Q21 

4.80 

5 

1.55 


50 

.35 



.54 


Q22 

5.00 

5 

1.09 


38 

.23 




-.35 

Q23 

5.33 

5 

1.01 

.43 

.30 

.36 




Q24 

4.74 

5 

1.07 

.50 

.38 




-.47 

Q25 

5.86 

6 

.89 

.54 

.47 

.68 




Q26 

5.41 

6 

1.20 

.52 

.39 


.54 



Q27 

4.97 

5 

1.42 

.45 

.24 



.33 


Q28 

5.70 

6 

1.12 

.58 

50 


.63 



Q29 

6.11 

6 

.82 

.58 

.49 

.65 




Q30 

5.73 

6 

.88 

.40 

.41 

.60 




Q31 

5.75 

6 

.90 

.59 

.43 

.56 




Q32 

5.04 

5 

1.29 

.49 

.60 


.78 



Q33 

5.92 

6 

.94 

.57 

.43 

.63 




Q34 

5.43 

6 

1.12 

.55 

.49 


.66 



Q35 

5.62 

6 

1.04 

.45 

.41 

.66 




Q36 

5.54 

6 

1.04 

.56 

.43 

.33 

.48 



Q37 

5.02 

5 

1.44 

.42 

.18 





Q38 

5.67 

6 

1.10 

.60 

.50 


.57 



Q39 

5.69 

6 

1.17 

.56 

.45 

.70 




Q40 

5.43 

6 

1.00 

.38 

.35 

.47 




Q41 

5.62 

6 

.97 

.53 

.39 

.61 




Q42 

5.29 

5 

1.21 

.35 

.26 


.39 



Q43 

6.07 

6 

.85 

.55 

.43 

.64 




Q44 

5.89 

6 

.88 

.54 

.36 

.42 

.34 



Q45 

4.42 

4 

1.36 

.32 

.30 


.35 

.38 


Q46 

5.44 

6 

1.14 

.42 

.25 

.36 




Q47 

4.85 

5 

1.27 

.45 

.22 

.38 




Q48 

5.83 

6 

1.00 

.48 

.29 

.38 
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Table 2- Coefficient Alpha Values for the Untransformed Data 


Cronbach’s Alpha 

Group 2 Sample (N 
= 502) 

Group 1 Sample 
(Seal et al. 2011b) 
(N = 632) 

48 items 

.91 

.91 

Aware 

.82 

.79 

Consider 

.83 

.81 

Connect 

.85 

.83 

Influence 

.82 

.84 


Table 3 - Four Factor Pearson Test-Retest Correlations 


Factor 

r 

P 

Mean 1 

Mean 2 

SD1 

SD 2 

Awareness 

.36 

< .01 

65.4 

61.1 

6.6 

7.4 

Consideration 

.53 

< .001 

65.5 

60.5 

8.0 

8.2 

Connection 

.79 

< .001 

62.5 

60.6 

10.7 

10.0 

Influence 

.74 

< .001 

57.4 

53.8 

10.6 

9.4 

Mean 

.61 


62.7 

59.0 

9.0 

8.8 


Table 4 - t-tests of Factor Score Mean Differences (time2 - timel) 

95% Confidence Interval 
2-tailed Lower Upper 

Factor _ t _ df _ p _ Bound_Bound 


Awareness 

Consider 


5.04 

5.73 


83 

83 


< .001 
< .001 


2.6 

3.2 


6.1 

6.6 


onnect 2.67 

83 

<.010 



0.5 

3.4 

lfluence 4.56 

83 

<.001 



2.0 

5.2 

Table 5 - 

Convergent/Discriminant Correlations 


MSCEIT N = 85 


AWARE 

CONSIDER 

CONNECT 

INFLUENCE 

Perceiving Emotions 


.08+ 

.15 


-.02+ 

-.03+ 

Using Emotions 


.11 + 

.04+ 


.15 

.11 + 

Understanding Emotions 


.05+ 

-.02+ 


-.11 + 

-.03+ 

Managing Emotions 


-.03+ 

-.01+ 


.00+ 

-.19 

EQ-i N = 280 


AWARE CONSIDER 

CONNECT 

INFLUENCE 

Intrapersonal 


.32* 

.19* 


.30* 

.31* 

Interpersonal 


.21* 

.20* 


.28* 

.15* 

Stress Management 


.22* 

.19* 


.14* 

.16* 

Adaptability 


.28* 

.23* 


.21* 

.23* 

General Mood 


.20* 

.15* 


.26* 

.25* 

IPIP N = 181 


AWARE CONSIDER 

CONNECT 

INFLUENCE 

Extraversion 


.31* 

.09 


.52* 

.46* 

Agreeable 


.31* 

.48* 


.41* 

.16* 

Conscientious 


.26* 

.38* 


.18* 

.09+ 

Emotional Stability 


.46* 

.33* 


.32* 

.20* 

Intellect 


.43* 

.33* 


.20* 

.29* 

TMMS N = 248 


AWARE CONSIDER 

CONNECT 

INFLUENCE 
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Attention 

.15* 

.20* 

.37* 

.19* 

Clarity 

.43* 

.36* 

.33* 

.22* 

Repair 

.31* 

.34* 

.34* 

.30* 

SM N = 209 

AWARE 

CONSIDER 

CONNECT 

INFLUENCE 

Modify Self 

.13 

.14* 

.07+ 

.13 

Sensitivity Others 

.15* 

.19* 

.07+ 

.05+ 

SD N = 239 

AWARE 

CONSIDER 

CONNECT 

INFLUENCE 

Social Desirability 

.06+ 

.18* 

.07+ 

.05+ 


*. Correlation is significant at the .05 level (2-tailed), indicating convergent validity. 

+. Correlation is not significant at the .20 level (2-tailed), indicating discriminant validity. 


Table 6 - MSCEIT Correlations 


MSCEIT N = 

85 

Perceiving 

Emotions 

Using Emotions 

Understanding 

Emotions 

Managing 

Emotions 

AWARE 

.08+ 

.11+ 

.05+ 

-.03+ 

CONSIDER 

.15 

.04+ 

-.02+ 

-.01 + 

CONNECT 

-.02+ 

.15 

-.11 + 

.00+ 

INFLUENCE 

-.03+ 

.11+ 

-.03+ 

-.19 


*. Correlation is significant at the .05 level (2-tailed), indicating convergent validity. 

+. Correlation is not significant at the .20 level (2-tailed), indicating discriminant validity. 


Table 7 - EQ 

■i 

Correlations 





Stress 



EQ-i N = 280 

Intrapersonal 

Interpersonal 


Management 

Adaptability 

General Mood 

AWARE 

.32* 

.21* 


.22 : 


.28* 

.20* 

CONSIDER 

.19* 

.20* 


.19 : 

£ 

.23* 

.15* 

CONNECT 

.30* 

.28* 


.14 : 

* 

.21* 

.26* 

INFLUENCE 

.31* 

.15* 


.16 : 

* 

.23* 

.25* 


*. Correlation is significant at the .05 level (2-tailed), indicating convergent validity. 

+. Correlation is not significant at the .20 level (2-tailed), indicating discriminant validity. 


Table 8 - IPIP Correlations 


IPIP N = 181 

Extraversion 

Agreeable 

Conscientious 

Emotional 

Stability 

Intellect 

AWARE 

.31* 

.31* 

.26* 

.46* 

.43* 

CONSIDER 

.09 

.48* 

.38* 

.33* 

.33* 

CONNECT 

.52* 

.41* 

.18* 

.32* 

.20* 

INFLUENCE 

.46* 

.16* 

.09+ 

.20* 

.29* 


*. Correlation is significant at the .05 level (2-tailed), indicating convergent validity. 

+. Correlation is not significant at the .20 level (2-tailed), indicating discriminant validity. 


Validation social emotional development, Page 19 









Research in Higher Education Journal 


Table 9 - TMMS Correlations 


TMMS N = 248 

Attention 

Clarity 

Repair 

AWARE 

.15* 

.43* 

.31* 

CONSIDER 

.20* 

.36* 

.34* 

CONNECT 

.37* 

.33* 

.34* 

INFLUENCE 

.19* 

.22* 

.30* 


*. Correlation is significant at the .05 level (2-tailed), indicating convergent validity. 

+. Correlation is not significant at the .20 level (2-tailed), indicating discriminant validity. 


Table 10 - Self-Monitoring Correlations 


SM N=209 

Modify Self 

Sensitivity Others 

AWARE 

.13 

.15* 

CONSIDER 

.14* 

.19* 

CONNECT 

.07+ 

.07+ 

INFLUENCE 

.13 

.05+ 


*. Correlation is significant at the .05 level (2-tailed), indicating convergent validity. 

+. Correlation is not significant at the .20 level (2-tailed), indicating discriminant validity. 

Table 11 - Marlowe-Crowne Correlations 



SD N=239 

Social Desirability 


\WARE 


.06+ 


CONSIDER 


.18* 


CONNECT 



INFLUENCE 


.05+ 


*. Correlation is significant at the .05 level (2-tailed), indicating convergent validity. 

+. Correlation is not significant at the .20 level (2-tailed), indicating discriminant validity. 
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